CLAIMS 

1. A light source estimation apparatus to correctly 
estimate the image-pickup light source, in which from sensor 
response values obtained upon pickup of an image of an 
unspecified arbitrary object, image-pickup means having a 
plurality of different spectral sensitivity, characteristics 
estimates spectral characteristics indicating color of an 
unknown image-pickup light source irradiating an object, 
comprising : 

storage means for storing, for each test light source, 
parameters for projecting said sensor response values into an 
evaluation space not dependent on said image-pickup light source 
by performing operations which can be colorimetrically 
approximated from a plurality of different said known spectral 
sensitivity characteristics of said image-pickup means and from 
the spectral characteristics of a plurality of test light 
sources assumed in advance; 

projection conversion means for projecting said sensor 
response values into said evaluation space not dependent on the 
image-pickup light source using parameters stored in said 
storage means; and, 

evaluation means for evaluating the correctness of said 

plurality of test light sources based on the image distribution 

state of sample values of an image scene projected by said 
projection conversion means. 



2. A light source estimation method in which from sensor 
response values obtained upon pickup of an image of an 
unspecified arbitrary object, image-pickup means having a 
plurality of different spectral sensitivity characteristics 
estimates spectral characteristics indicating color of an 
unknown image-pickup light source irradiating an object, 
comprising the steps of: 

projecting said sensor response values into an evaluation 
space not dependent on the image-pickup light source through 
operations which can be colorimetrically approximated from known 
spectral sensitivity characteristics of image-pickup means and 
from spectral characteristics of an assumed test light source; 
and 

estimating the correct image-pickup light source by 
evaluating the correctness of a plurality of said test light 
sources based on a state of distribution of sampled values of 
the projected scene. 

3. The light source estimation method according to claim 

2, 

wherein a vector space for said evaluation is a space in 
which weighting coefficients used to approximate the spectral 
reflectivity of diverse object surfaces by conjoining a 
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plurality of reflectivity basis functions, represent the 
spectral reflectivity characteristics specific to an object 
surface, or a space in which the weighting coefficients become 
further converted values through fixed operations. 

4. The light source estimation method according to claim 

3, 

wherein said reflectivity basis functions to approximate 
the spectral reflectivity are spectral reflectivity components 
obtained by statistical analysis of the spectral reflectivity 
data of a plurality of known object surfaces as a population; 
are intentionally extracted spectral reflectivity components; or 
are a combination of both. 

5. The light source estimation method according to claim 

2, 

wherein a vector space for said evaluation is a space in 
which spectral distribution values for light reflected on an 
object surface from a single virtual reference light source 
having a specific spectral distribution are converted into a 
plurality of channels by fixed operations. 



6. The light source estimation method according to claim 



wherein a reference light source, said spectral 
distribution of which is fixed over a wavelength range, is used. 

7. The light source estimation method according to claim 

2, 

wherein a plurality of light sources with different known 
spectral distributions are taken to be said test light sources; 
spectral distribution data for each test light source or 
coefficients for computation corresponding to each test light 
source to which said spectral distribution data is applied are 
stored in advance; and the data or coefficients are referenced 
at the time of said light source estimation. 

8. The light source estimation method according to claim 

2, 

wherein a plurality of different representative light 
sources are extracted and stored in advance as said test light 
sources from among spectral distribution data for various known 
light sources, from among coefficients to approximate the 
spectral distribution data by weighted linear sums of a 
plurality of light source basis functions, or from among indexes 
obtained using fixed computation formula from the spectral 
distribution data; and the spectral distribution data of each of 
the test light sources, or the computation coefficients 
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corresponding to each of the test light sources to which the 
data is applied, are referenced at the time of said light source 
estimation. 

9. The light source estimation method according to claim 

8, 

wherein, as the information for the plurality of different 
light sources stored in advance, spectral distribution data for 
a specific light source or computation coefficients 
corresponding to a specific light source to which • the 
distribution data is applied are used; and said plurality of 
test light sources are generated and referenced by appropriate 
selection, interpolation processing, or the like at the time of 
said light source estimation. 

10. The light source estimation method according to claim 

7, 

wherein said plurality of representative test light sources 
are categorizable by the color temperature value of the light 
source, by the physical light emission method of the light 
source, or by both. 



11. The light source estimation method according to claim 



wherein, among the sensor response values of said image- 
pickup means, values with respect to all pixels or values with 
respect to pixels sampled at appropriate positions, in 
appropriate ranges, and at appropriate intervals within the 
spatial position of the image-pickup plane are used. 

12. The light source estimation method according to claim 

2, 

wherein, among the sensor response values of said image- 
pickup means, values with respect to only pixels the values for 
each channel of which are in a specified range, or values with 
respect to all pixels other than pixels the values for each 
channel of which are in a specified range are used. 

13. The light source estimation method according to claim 

2, 

wherein, at the time of projecting sensor response values 
of said image-pickup means into evaluation space, or prior to 
said time, scaling is performed at a fixed arbitrary ratio or at 
an appropriate ratio determined in advance according to image- 
pickup results. 



14. The light source estimation method according to claim 



wherein sensor response values of said image -pickup means 
are used after adding noise, exposure error, or other temporally 
fluctuating quantities supposed- in said image-pickup means, or 
after adding pixels to which such fluctuating quantities have 
been added. 

15. The light source estimation method according to claim 

2, 

wherein with respect to each of said test light sources, a 
statistical quantity obtained from values of sample pixels 
projected into evaluation space; a statistical quantity obtained 
from an image distribution indicating the frequency distribution 
in evaluation space generated from sample pixels; a statistical 
quantity obtained from the image color gamut indicating the 
region in the evaluation space in which sample pixels are 
distributed; or a combination of any two or more of these, are 
used, either without further modification, or after conversion 
into values by a fixed operation, as said estimation criterion 
for an index of correctness assumed in advance. 

16. The light source estimation method according to claim 

2, 

wherein a statistical quantity obtained from sample pixels 
in sensor space of said sensor response values, or a statistical 



quantity obtained from values converted by a fixed operation 
from sample pixel -values in sensor space, are projected into 
evaluation space with respect to each of said test light sources, 
and are used, either without further modification, or after 
conversion into values by a fixed operation, as said estimation 
criterion for an index of correctness assumed in advance. 

17. The light source estimation method according to claim 

15, 

wherein, with respect to the spectral reflectivity of an 
object surface, an index of the correctness of each of said test 
light sources is calculated in advance using statistical 
quantities added constraints or weighting formable in a specific 
region of the evaluation space, based on the physical 
possibility in the range from 0 to 1 at each wavelength and on 
an assumed probabilistic distribution in the real world in which, 
on average, there exist ' numerous surface approximating 
achromaticity with a flat wavelength characteristic. 

18. The light source estimation method according to claim 

2, 

wherein, with respect to each of said test light sources, a 
correlation function of a reference color gamut recorded in 
advance, ref erenceable, and indicating the range of appearance 



in the evaluation space; with sample pixel values projected into 
the evaluation space; with the frequency distribution in the 
evaluation space generated from the sample pixels; with the 
region in the evaluation space in which sample pixels are 
distributed; or with a combination of any two or more of same, 
is used as an index of said estimation criterion. 

19. The light source estimation method according to claim 

18, 

wherein a weighting distribution and region information 
generated from the frequency distribution in the evaluation 
space of values converted from spectral reflectivity data of 
various object surfaces into coefficients approximated by 
reflectivity basis functions, or of values obtained by 
converting said coefficients by a fixed operation, are used as 
said reference color gamut. 

20. The light source estimation method according to claim 

18, 

wherein a weighting distribution and region information 
generated from the frequency distribution of values, in which 
sensor response values which are either the result of image 
pickup of a variety of actually existing scenes or the result of 
predicting by numerical operations the images picked up for a 



variety of virtual scenes are projected into evaluation space 
for each scene using operations capable of colorimetrically 
approximating from spectral sensitivity characteristics of said 
image-pickup means and from spectral distribution 
characteristics of the image-pickup light source measured at the 
time of image pickup of each scene, are used as said reference 
color gamut. 

21. The light source estimation method according to claim 

18, 

wherein, with respect to spectral reflectivity of an object 
surface, a weighting distribution and region information 
generated from a frequency distribution determined based on a 
physical possibility in the range 0 to 1 at each wavelength and 
on an assumed probabilistic distribution in the real world in 
which, on average, there exist numerous surface approximating 
achromaticity with a flat wavelength characteristic, are used as 
said reference color gamut. 

22. The light . source estimation method according to claim 

19, 

wherein, before or after generating said reference color 
gamut from any of said frequency distributions or from a 
combination thereof, with respect to the distribution in 



evaluation space, interpolation, extrapolation, removal, spatial 
filtering, or other processing according to fixed criteria are 
performed . 

23. The light source estimation method according to claim 

15, 

wherein, in generation of an index of correctness for each 
of said test light sources, in order to emphasize the high color 
saturation region in which the difference between test light 
sources appears more prominently in the evaluation space, the 
image distribution is extracted and weighted operations are 
performed on the outline or in the vicinity thereof in the image 
color gamut. 

24. The light source estimation method according to claim 

15, 

wherein, with respect to the image distribution or image 
region of sample pixels projected into the evaluation space, 
after performing interpolation, extrapolation, removal, spatial 
filtering, or other processing according to fixed criteria, an 
index of correctness is calculated for each of said test light 
sources . 
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25. The light source estimation method according to claim 

15, 

wherein a plurality of different indexes generated from 
sample pixel values projected into a single evaluation space, or 
a plurality of different indexes generated from sample pixel 
values projected into a plurality of different evaluation spaces, 
are conjoined by numeric means; are selected by conditional 
branching; or are both combined, to generate a new index used to 
evaluate the correctness of each of said test light sources. 

26. The light source estimation method according to claim 

2, 

wherein the test light source having the highest index of 
correctness among said plurality of test light sources is 
determined as the estimated light source. 



27. The light source estimation method according to claim 

2, 

wherein the weighted averages of two or more light sources 

having high correctness among said plurality of test light 

sources is determined as the estimated light source. 

28. The light source estimation method according to claim 

26, 
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wherein a process, in which the light source with the 
highest index of correctness among said plurality of test light 
sources is initially selected, and different light sources 
obtained in finely divided . vicinity of said selected light 
source are referenced to generate indexes of correctness for 
each light source, is repeated. 

29. The light source estimation method according to claim 

26, 

wherein said test light sources include two or more 
categories according to physical light emission method; color 
temperature judgment processing based on an index indicating 
that, within each category, the color temperature is closest to 
the color temperature of the image-pickup light source, and 
light emission method judgment processing based on an index 
indicating that the light source is closest to the physical 
light emission method of the image-pickup light source, using 
the same or another index, are performed; and the estimated 
light source is determined from both the judgment results. 

30. The light source estimation method according to claim 

26, 

wherein said test light sources include two or more 
categories according to physical light emission method, and the 



estimated light source is determined based on an index 
indicating a light source closest to the image-pickup light 
source, with respect only to a test light sources belonging to a 
category specified by the user or to a category provided by 
category judgment means differing from said estimation means. 

31. The light source estimation method according to claim 

2, 

wherein the image-pickup light source determined by said 
estimation and a light source determined by an estimation method 
different from said estimation are conjoined by numeric means, 
are selected by conditional branching, or are both combined, to 
determine the final estimated light source. 

32. An image-pickup apparatus in which from sensor 
response values obtained upon pickup of an image of an 
unspecified arbitrary object, image-pickup means having a 
plurality of different spectral sensitivity characteristics 
estimates spectral characteristics indicating color of an 
unknown image-pickup light source irradiating an object, and 
which uses, in color balance processing of the sensor response 
of said image-pickup means, the spectral characteristics which 
are the color of the estimated light source or parameters 
appropriate thereto, comprising: 
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storage means for storing, for each test light source, 
parameters for projecting said sensor response values into an 
evaluation space not dependent on said image-pickup light source 
by performing operations which can be colorimetrically 
approximated from a plurality of different said known spectral 
sensitivity characteristics of said image-pickup means and from 
the spectral characteristics of a plurality of test light 
sources assumed in advance; 

projection conversion means for projecting said sensor 
response values into said evaluation space not dependent on the 
image-pickup light source using parameters stored in said 
storage means; 

evaluation means for evaluating the correctness of said 
plurality of test light sources based on the image distribution 
state of sample values of an image scene projected by said 
projection conversion means; 

light source estimation means for estimating the final 
image-pickup light source to be determined as the estimated 
light source by conjoining in numerical formulas, by selecting 
through conditional branching, or by combining both of, an 
image-pickup light source determined by said estimation and a 
light source determined by an estimation method different from 
said estimation; and 

color balance adjustment means which uses spectral 
characteristics, which are the color of the estimated image- 



pickup light source, or parameters appropriate thereto in color 
balance processing of the sensor response of said image-pickup 
means . 

33. An image processing method in which from sensor 
response values obtained upon pickup of an image of an 
unspecified arbitrary object , image-pickup means having a 
plurality of different spectral sensitivity characteristics 
estimates spectral characteristics indicating color of an 
unknown image-pickup light source irradiating an object; and 
which uses the spectral characteristics, which are the color of 
the estimated light source, or parameters appropriate thereto in 
color balance processing of the sensor response of said image- 
pickup means, comprising the steps of: 

projecting said sensor response values into an evaluation 
space not dependent on the image-pickup light source through 
operations which can . be colorimetrically approximated from 
spectral sensitivity characteristics of known image-pickup means 
and from spectral characteristics of an assumed test light 
source; 

estimating the correct image-pickup light source by 
evaluating the correctness of a plurality of said test light 
sources based on a state of distribution of sampled values of 
the projected scene; 
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estimating the final image-pickup light source to be 
determined as the estimated light source by conjoining using 
numeric means, by selecting using conditional branching, or by 
both combined, the image-pickup light source determined by said 
estimation, and a light source' determined by an estimation 
method different from said estimation; and 

using, in color balance processing of the sensor response 
of said image-pickup means, spectral characteristics which are 
the color of the estimated image-pickup light source, or 
parameters appropriate thereto. 
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